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(57) Abstract 



A controller, in accordance with a preferred embodiment, provides, in a specified region of first television picture generated by a 
first digitally compressed video signal, an overlay of a second picture embodied in a second video signal. The controller includes an 
I-frame buffer for storing updated macroblock pixel data in I -frame format derived from the second signal. Hie controller also has a matte 
mask storage register for storing data that identifies the specified region by means of macroblocks. A matte substitution processor is in 
communication with the matte mask storage register and has as an input the first signal and provides an overlayed output; the controller 
inhibits transmission to the output of any macroblock, of the first signal's data stream, tying in the specified region. It also functions to 
substitute for such macroblock a corresponding macroblock from the I-frame buffer. Further embodiments permit overlay of a graphic 
display for interactive television on a picture derived from a digitally compressed video signal. 
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COMPRESSED DIGITAL OVERLAY CONTROLLER AND METHOD 

5 Technical Field 

The present invention relates to controllers for 
providing in a first television picture generated by signals 
in a compressed digital format, such as under the MPE62 
standard, an overlay of a second picture. 

10 Background Art 

Overlay controllers for analog television signals are 
veil known in the art. The MPEG 2 compression standard for 
digital video signals also contemplates the possibility of 
overlaying a first compressed digital video signal with a 

15 second compressed digital video signal. A controller for 
achieving the overlay of two such digitally compressed video 
signals is inherently complex. In some instances, it is 
desirable to overlay a compressed digital video signal (such 
as, for example, a movie or other material made available to 

20 a cable television subscriber) with a relatively slowly 
changing analog signal (such as, for example, a display 
including a movable cursor and a number of selection buttons 
to permit interactive cable television service to a 
subscriber operating a remote control unit causing movement 

25 of the cursor and operation of the buttons) • In such a 

case, the art currently would require, before overlaying the 
signal, either (i) A-to-D conversion of the analog signal 
and its subsequent compression or (ii) decompression said D- 
to-A conversion of the digital signal. Such techniques are 

30 inefficient and expensive. 

nummary of the Invention 
In accordance with a preferred embodiment of the 
invention there is provided a controller, for providing, in 
a specified region of first television picture generated by 

35 a first digitally compressed video signal, an overlay of a 
second picture embodied in a second video signal. The 
controller includes an I-frame buffer for storing updated 
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macroblock pixel data in I-frame format derived from the 
second signal. The controller also has a matte mask storage 
register for storing data that identifies the specified 
region by means of macroblocks. A matte substitution 
5 processor is in communication with the matte mask storage 
register and has as an input the first signal and provides 
an overlayed output; the controller inhibits transmission to 
the output of any macroblock, of the first signal's data 
stream, lying in the specified region. It also functions to 

10 substitute for such macroblock a corresponding macroblock 

from the I-frame buffer. Further embodiments permit overlay 
of a graphic display for interactive television on a picture 
derived from a digitally compressed video signal. 

Brief Description of the Drawings 

15 The invention will be more readily understood by 

reference to the following description taken with the 
accompanying drawings in which: 

Fig. 1 is a diagram of the effect of an overlay in 
accordance with a preferred embodiment of the invention; 

20 Fig. 2 is a diagram of a matte mask used to define the 

boundaries of the overlay in Fig. 1; 

Fig. 3 is a block diagram of an embodiment of the 
invention for achieving the overlay of Fig. 1; and 

Fig. 4 provides detail of certain portions of Fig. 3. 

25 Detailed Description fif Specif i g ttmHodimenta 

Fig. 1 illustrates the effect of an overlay of a first 
MPEG2 encoded digital video source, which is referred to 
below as source 1, by a second signal, which is referred to 
below as source 2. In the context of interactive 

30 television, source 1 may be, for example, a digitally 

compressed movie or other information service (as that term 
is defined above) . Source 2 may be, for example, a graphic 
display, in analog format, showing a menu of choices 
available to a subscriber and a cursor movable by the 

35 subscriber (using a remote control) , so arranged as to 

permit the subscriber to communicate interactively with the 
headend and to make a selection from the menu of choices. 



WO 95/33342 



PCT/US95/06924 



- 3 - 

Thus, source 1 could be a movie preview, for example, and 
source 2 could present a menu of choices to the subscriber 
permitting selection of the movie or an opportunity to 
preview another movie, and so on. In this example, source 1 
5 is shown in region A of Fig. 1 and source 2 is shown in 
region B. 

In accordance with MPEG 2 encoding, successive 
contiguous 16 X 16 square arrays (herein called 
"macroblocks") of pixels are identified over the entire 

10 picture area; a sample macroblock is shown as item 11 in 
Fig. 1. Each macroblock in the picture area is given a 
unique numerical address, in the general manner illustrated 
in Fig. 1, corresponding to its location in the picture. 
The boundary between regions A and B in Fig. 1 is 

15 defined by a matte mask, shown in Fig. 2, that specifies the 
macroblock addresses that are included (or, alternatively, 
excluded) in region A. 

Fig. 3 is a block diagram for an embodiment of the 
present invention for achieving the overlay described above. 

20 Inputs 30 include the digitally compressed source 1 signal 
as well as data showing activity of the subscriber's remote 
unit. (Additional video data may be provided for use in 
creating the video for region B of Fig. 1. Such additional 
data may come as digitally compressed data from the inputs 

25 30 and be decompressed by decompressor 33, and/or it may 

come from other source 34, which may be a decompressor or TV 
tuner. Data from the subscriber's remote unit is processed 
by CPU 31, operating with RAM 35, and graphics overlay 32 
provides an analog input to mixer-encoder 36. This input 

30 321 shows cursor position and the menu choices available to 
the subscriber for interaction with the headend. The mixer- 
encoder 36 first mixes input 321 with inputs from sources 33 
and 34, and then encodes the mixed analog signal as a series 
of I-frames in MPEG2 format. The I-frame encoded output of 

35 mixer-encoder 36 is provided to MPEG 2 mixer 37, the output 
of which is subjected to regulation by flow controller 38 to 
prevent excessively high data rates from appearing at the 
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output 39. This regulation is necessary to prevent the data 
rates from exceeding the capacity of the distribution 
system, and, particularly, of the subscriber's equipment. 

Detail of the mixer-encoder 36, the MPEG2 mixer 37, and 
5 the flow controller 38 is presented in Fig. 4. The MPEG2 
encoded digital video data stream identified as source 1 in 
Fig. 4 is here overlayed with a desired signal output, 
identified as source 2, from the video effects and mixer 
module 41. in the manner described herein. In this example, 

10 source 2 is in analog YUV format. The source 2 signal is 
then digitally encoded by encoder 42 as a series only of I- 
frames in MPE62 format. (That is, no B-frames or P- frames 
are provided in the output from encoder 32. It will be 
appreciated that the picture content of region B is 

15 generally expected to be relatively slowly changing; the 
changes most commonly expected are primarily those 
associated with cursor motion and selection of choices by 
the subscriber. Accordingly, the full MPEG encoding of 
source 2 is not necessary. 

20 Moreover, constraints on the system are required in 

order for it to provide appropriate responsiveness. Each 
frame duration is 1/3 oth of a second. Encoding source 2 as 
a series of I-frames will therefore mean up to l/30th of a 
second until an action on the subscriber's remote unit 

25 (excluding other sources of delay) will have been encoded by 
encoder 42. Moveover, in addition to the time for encoding 
here by encoder 42, there must also be taken into account an 
equal time for decoding for display on the subscriber's 
television; thus the total delay is double the time for 

30 encoding. It is possible, for example, to use P frames 

along with I frames, but if P frames are used in addition to 
I frames, the encoding time will be doubled. 

The I-frame output of encoder 42 is stored in I-frame 
buffer 33, the data in which is updated as necessary based 

35 on output from the encoder 42. Any of the macroblocks in 
the I-frame buffer that do not change do not need to be 
updated, and where there is little change in source 2 video 
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there will be relatively infrequent updating of the buffer 
42. 

The I- frame data in buffer 43 is selectively inserted 
in the source 1 MPEG 2 encoded data stream in the following 
5 manner. The source 1 data stream is run through macroblock 
detector 46 to detect macroblocks, and its output is 
provided to the matte substitution processor 44. Matte mask 
storage register 45 stores the addresses of the macroblocks 
constituting the matte mask of Fig. 2, and provides data to 

10 the matte processor 44. Whenever the processor identifies a 
macroblock (regardless whether I-, B-, or P-frame) from 
detector 46 that lies outside (or, alternatively, inside) 
the matte region defined in register 45, it substitutes an 
I-frame macroblock of the corresponding address from I-f rame 

15 buffer 43. 

It is possible , if there is a great deal of change in 
the picture content of source 2, then the output 39 of 
system would be at a data rate beyond system capacity. In 
such a case, the processor 44 will continue to block source 
20 1 macroblocks lying outside the matte region, but will 

refrain from substituting data from the I-frame buffer 43 to 
the extent necessary to slow the data rate at the output to 
a rate within system limits. 
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What is claimed is: 

1. A controller, for providing, in a specified region 
of a television picture generated by a first digitally 
compressed video signal, an overlay of a second picture 

5 embodied in a second video signal, comprising: 

an I-frame buffer for storing updated macroblock pixel 
data in I-frame format derived from the second signal; 

matte mask storage register for storing data that 
identifies the specified region by means of macroblocks; 
10 matte substitution processor, in communication with the 

matte mask storage register and having as an input the first 
signal and providing an overlayed output, for inhibiting 
transmission to the output of any macroblock, of the first 
signal's data stream, lying in the specified region and for 
15 substituting for such macroblock a corresponding macroblock 
from the I-frame buffer. 

2. A controller according to claim 1, wherein the 
second video signal is in an analog format, further 
comprising : 

20 an I-frame encoder for encoding the second signal as a 

series of I-frames, the I-frame encoder having an output in 
communication with the I-frame buffer. 

3. A controller according to claim 2, further 
comprising: 

25 a macroblock detector in communication with the first 

signal and the matte substitution processor, for detecting 
the presence of a macroblock. 

4. A controller according to claim 1, wherein the 
matte substitution processor includes flow control means for 

30 refraining from substituting data from the I-frame buffer, 
while continuing to inhibit transmission of any macroblock 
of the first signal's data stream lying in the specified 
region, to the extent necessary to slow the data rate at the 
overlayed output to a rate within system limits. 

35 5. A controller according to claim 3, wherein the 

matte substitution processor includes flow control means for 
refraining from substituting data from the I-frame buffer, 
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while continuing to inhibit transmission of any macroblock 
of the first signal's data stream lying in the specified 
region, to the extent necessary to slow the data rate at the 
overlayed output to a rate within system limits. 
5 6. An interactive television system providing, in a 

specified region of a television picture generated by a 
first digitally compressed video signal, an overlay of a 
graphic display permitting user inp^t via a remote control, 
such graphic display embodied in a second signal in an 

10 analog format, the system comprising: 

an I-frame encoder for encoding the second signal as a 
series of I-frames; 

an I-frame buffer for storing updated macroblock pixel 
data in I-frame format from the I-frame encoder; 

15 matte mask storage register for storing data that 

identifies the specified region by means of macroblocks; 

matte substitution processor, in communication with the 
matte mask storage register and having as an input the first 
signal and providing an overlayed output, for inhibiting 

20 transmission to the output of any macroblock, of the first 
signal's data stream, lying in the specified region and for 
substituting for such macroblock a corresponding macroblock 
from the I-frame buffer. 

7. A controller according to claim 6, further 
25 comprising: 

a macroblock detector in communication with the first 
signal and the matte substitution processor, for detecting 
the presence of a macroblock. 

8. A controller according to claim 6, wherein the 

30 matte substitution processor includes flow control means for 
refraining from substituting data from the I-frame buffer, 
while continuing to inhibit transmission of any macroblock 
of the first signal's data stream lying in the specified 
region, to the extent necessary to slow the data rate at the 

35 overlayed output to a rate within system limits. 

9. A controller according to claim 7, wherein the 
matte substitution processor includes flow control means for 
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refraining from substituting data from the I-frame buffer, 
while continuing to inhibit transmission of any macroblock 
of the first signal's data stream lying in the specified 
region, to the extent necessary to slow the data rate at the 
5 overlayed output to a rate within system limits. 

10. A method of overlaying in a specified region of a 
television picture a first digitally compressed video signal 
with a second analog video signal, comprising: 

encoding the second signal as a series of I-frames; 
10 storing the encoded I-frame macroblock pixel data in an 

I-frame buffer; 

storing data that identifies the specified region by 
means of macroblocks in a matte mask storage register; 
processing the data stream of the first signal and 
15 providing an overlayed output, by inhibiting transmission to 
the output of any macroblock, of the first signal's data 
stream, lying in the specified region and substituting for 
such macroblock a corresponding macroblock from the I-frame 
buffer. 

20 11. A method according to claim 10, further 

comprising: 

refraining from substituting data from the I-frame 
buffer, while continuing to inhibit transmission of any 
macroblock of the first signal's data stream lying in the 
25 specified region, to the extent necessary to slow the data 
rate at the overlayed output to a rate within system limits. 

12. A method of providing an interactive television 
system having, in a specified region of a television picture 
generated by a first digitally compressed video signal, an 
30 overlay of a graphic display permitting user input via a 
remote control, such graphic display embodied in a second 
signal in an analog format, the method comprising: 

encoding the second signal as a series of I-frames; 

storing the encoded I-frame macroblock pixel data in an 
35 I-frame buffer; 

storing data that identifies the specified region by 
means of macroblocks in a matte mask storage register; 
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processing the data stream of the first signal and 
providing an overlayed output, by inhibiting transmission to 
the output of any macroblock, of the first signal's data 
stream, lying in the specified region and substituting for 
5 such macroblock a corresponding macroblock from the I-frame 
buffer. 

13. A method according to claim 12, further 
comprising: 

refraining from substituting data from the I-frame 
10 buffer, while continuing to inhibit transmission of any 
macroblock of the first signal's data stream lying in the 
specified region, to the extent necessary to slow the data 
rate at the overlayed output to a rate within system limits. 
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